stream: fix 0 transform hwm backpressure #43685

ronag · 2022-07-05T08:58:47Z

Refs: #42457
Refs: https://github.com/nodejs/node/pull/43648/files

This also fixes a long standing issue where setting a hwm on a transform stream actually causes twice as much buffering as expected.

nodejs-github-bot · 2022-07-05T08:58:52Z

Review requested:

@nodejs/streams

ronag · 2022-07-05T08:59:29Z

@kanongil

mcollina

lgtm

kanongil · 2022-07-05T12:20:42Z

This patch does not fix my issue since it removes the ability to defer the call to _read() / _transform() by waiting to call read().

It seems you have modified the test code I provided here to suit your patch, by performing an immediate read() after the first read(). My original test code (that works on node v14 and node v16.3) will fail with this patch!

kanongil · 2022-07-05T12:24:34Z

Incidentally, the actual test that was failing in my code, will work with this patch, since I explicitly pass readableHighWaterMark: 0. Though the resulting stream would fail your drain test, since it doesn't pre-fetch.

ronag · 2022-07-05T12:35:58Z

This patch does not fix my issue since it removes the ability to defer the call to _read() / _transform() by waiting to call read().

It seems you have modified the test code I provided here to suit your patch, by performing an immediate read() after the first read(). My original test code (that works on node v14 and node v16.3) will fail with this patch!

I'm not sure I understand. Node streams are greedy so I don't think deferring a read is actually part of the design. Also this is a separate issue than what I'm trying to solve, which is the lack of back pressure.

Making a lazy transform by passing 0 is IMHO a separate issue and something I need to further consider.

kanongil · 2022-07-05T13:49:25Z

The lazy reads used to work. Whether it is by design or not.

Anyway, you are correct that this fixes the backpressure issue for highWaterMark: 0, and at most 1 element should be buffered using this patch. It does not fix it for writableHighWaterMark: 0, readableHighWaterMark: 0.

kanongil · 2022-07-05T15:01:23Z

Just because you haven't thought that transfrom streams can be lazy read, it has worked so far, and makes sense given how it is documented. It is also used internally for process.stdin TTY afaik, where you don't want to read user input greedily.

At the very least, this pre-load behaviour makes it inconsistent with how a standard Readable works. Eg. for

const r = new Readable({
  highWaterMark: 0,
  read() {
    console.log('_read');
    process.nextTick(() => this.push('a'))
  }
});

If you do a single r.read() between each nextTick it will alternate between returning the pushed value, and scheduling a new _read().

Also, removing the feature mid-release cycle (node v16.3 => v.16.4) seems like an error.

benjamingr

Apologies to the ecosystem for changing everyone's buffering in code using Node in advance.

lib/internal/streams/transform.js

kanongil · 2022-07-05T15:55:31Z

Apologies to the ecosystem for changing everyone's buffering in code using Node in advance.

I guess we were warned:

node/doc/api/stream.md

Lines 123 to 124 in b993789

    
           The mechanics of the internal buffering are an internal implementation detail 
        
           and may be changed at any time. However, for certain advanced implementations,

benjamingr · 2022-07-05T15:58:32Z

@kanongil lucky for us users always read the docs, only use public APIs and always use them in ways they expect :D

ronag · 2022-07-05T17:22:53Z

Just to clarify. I'm not saying we shouldn't fix the lazy loading. But this PR is not intended to fix that and we can try to fix that in a separate PR. This is ONLY trying to fix the back pressure issue.

nodejs-github-bot · 2022-07-11T09:54:10Z

CI: https://ci.nodejs.org/job/node-test-pull-request/45298/

nodejs-github-bot · 2022-07-12T21:41:43Z

CI: https://ci.nodejs.org/job/node-test-pull-request/45371/

nodejs-github-bot · 2022-07-14T17:53:10Z

CI: https://ci.nodejs.org/job/node-test-pull-request/45441/

nodejs-github-bot · 2022-07-17T15:50:44Z

CI: https://ci.nodejs.org/job/node-test-pull-request/45494/

mcollina

lgtm

nodejs-github-bot · 2022-07-18T10:39:40Z

Commit Queue failed

- Loading data for nodejs/node/pull/43685
✔  Done loading data for nodejs/node/pull/43685
----------------------------------- PR info ------------------------------------
Title      stream: fix 0 transform hwm backpressure (#43685)
Author     Robert Nagy  (@ronag)
Branch     ronag:fix-42457 -> nodejs:main
Labels     stream, author ready, commit-queue-squash
Commits    16
 - stream: fix 0 transform hwm backpressure
 - stream: add 0 hwm test
 - fixup! stream: fix 0 transform hwm backpressure
 - fixup! stream: fix 0 transform hwm backpressure
 - fixup! stream: fix 0 transform hwm backpressure
 - fixup! stream: fix 0 transform hwm backpressure
 - fixup! stream: fix 0 transform hwm backpressure
 - fixup! stream: fix 0 transform hwm backpressure
 - fixup! stream: fix 0 transform hwm backpressure
 - fixup! stream: fix 0 transform hwm backpressure
 - fixup! stream: fix 0 transform hwm backpressure
 - fixup! stream: fix 0 transform hwm backpressure
 - fixup! stream: fix 0 transform hwm backpressure
 - fixup! stream: add 0 hwm test
 - fixup
 - fixup: allow overriding writableHighWaterMark
Committers 1
 - Robert Nagy 
PR-URL: https://github.com/nodejs/node/pull/43685
Refs: https://github.com/nodejs/node/issues/42457
Refs: https://github.com/nodejs/node/pull/43648/files
Reviewed-By: Matteo Collina 
Reviewed-By: Benjamin Gruenbaum 
------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/43685
Refs: https://github.com/nodejs/node/issues/42457
Refs: https://github.com/nodejs/node/pull/43648/files
Reviewed-By: Matteo Collina 
Reviewed-By: Benjamin Gruenbaum 
--------------------------------------------------------------------------------
   ℹ  This PR was created on Tue, 05 Jul 2022 08:58:47 GMT
   ✔  Approvals: 2
   ✔  - Matteo Collina (@mcollina) (TSC): https://github.com/nodejs/node/pull/43685#pullrequestreview-1041623843
   ✔  - Benjamin Gruenbaum (@benjamingr): https://github.com/nodejs/node/pull/43685#pullrequestreview-1028862249
   ✖  Last GitHub CI failed
   ℹ  Last Full PR CI on 2022-07-17T15:50:44Z: https://ci.nodejs.org/job/node-test-pull-request/45494/
- Querying data for job/node-test-pull-request/45494/
   ✔  Last Jenkins CI successful
--------------------------------------------------------------------------------
   ✔  Aborted `git node land` session in /home/runner/work/node/node/.ncu

https://github.com/nodejs/node/actions/runs/2689876942

mcollina · 2022-07-18T10:49:32Z

git node land is failing but I do not understand why. What should I do in this case?

aduh95 · 2022-07-24T11:54:18Z

git node land is failing but I do not understand why. What should I do in this case?

Land manually.

aduh95 · 2022-07-24T11:55:21Z

Landed in 31d9edc

PR-URL: #43685 Refs: #42457 Refs: https://github.com/nodejs/node/pull/43648/files Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]>

##### [\`v18.7.0\`](https://github.com/nodejs/node/releases/tag/v18.7.0) ##### Notable changes - **doc**: - add F3n67u to collaborators (Feng Yu) [#43953](nodejs/node#43953) - deprecate coercion to integer in process.exit (Daeyeon Jeong) [#43738](nodejs/node#43738) - **(SEMVER-MINOR)** deprecate diagnostics_channel object subscribe method (Stephen Belanger) [#42714](nodejs/node#42714) - **events**: - **(SEMVER-MINOR)** expose CustomEvent on global with CLI flag (Daeyeon Jeong) [#43885](nodejs/node#43885) - **(SEMVER-MINOR)** add `CustomEvent` (Daeyeon Jeong) [#43514](nodejs/node#43514) - **http**: - **(SEMVER-MINOR)** add drop request event for http server (theanarkh) [#43806](nodejs/node#43806) - **lib**: - **(SEMVER-MINOR)** improved diagnostics_channel subscribe/unsubscribe (Stephen Belanger) [#42714](nodejs/node#42714) - **util**: - **(SEMVER-MINOR)** add tokens to parseArgs (John Gee) [#43459](nodejs/node#43459) ##### Commits - \[[`0aa255ab72`](nodejs/node@0aa255ab72)] - **bootstrap**: handle snapshot errors gracefully (Joyee Cheung) [#43531](nodejs/node#43531) - \[[`0783ddf57e`](nodejs/node@0783ddf57e)] - **buffer**: do not leak memory if buffer is too big (Keyhan Vakil) [#43938](nodejs/node#43938) - \[[`12657accdd`](nodejs/node@12657accdd)] - **build**: add .gitattributes for npm and other shims (Hrishikesh Kadam) [#43879](nodejs/node#43879) - \[[`c2db4f4581`](nodejs/node@c2db4f4581)] - **build**: make GitPod less noisy (Rich Trott) [#43829](nodejs/node#43829) - \[[`364deeadcd`](nodejs/node@364deeadcd)] - **build**: add GitHub token permissions for workflows (Varun Sharma) [#43743](nodejs/node#43743) - \[[`8b83b4d5be`](nodejs/node@8b83b4d5be)] - **child_process**: do not need to count length when maxBuffer is Infinity (theanarkh) [#43822](nodejs/node#43822) - \[[`c1893b7a7c`](nodejs/node@c1893b7a7c)] - **child_process**: avoid repeated calls to `normalizeSpawnArguments` (木杉) [#43345](nodejs/node#43345) - \[[`7b276b89b9`](nodejs/node@7b276b89b9)] - **cluster**: send connection to other server when worker drop it (theanarkh) [#43747](nodejs/node#43747) - \[[`e8c66f92a5`](nodejs/node@e8c66f92a5)] - **crypto**: remove unneeded guard (Rich Trott) [#43856](nodejs/node#43856) - \[[`c95132e9ea`](nodejs/node@c95132e9ea)] - **deps**: cherry-pick [libuv/libuv@`3a7b955`](libuv/libuv@3a7b955) (Ben Noordhuis) [#43950](nodejs/node#43950) - \[[`cc8d5426d2`](nodejs/node@cc8d5426d2)] - **deps**: cherry-pick [libuv/libuv@`abb109f`](libuv/libuv@abb109f) (Ben Noordhuis) [#43950](nodejs/node#43950) - \[[`7762e463d6`](nodejs/node@7762e463d6)] - **deps**: update corepack to 0.12.1 (Node.js GitHub Bot) [#43965](nodejs/node#43965) - \[[`1256c4dad5`](nodejs/node@1256c4dad5)] - **deps**: update hast-util-raw (Moshe Atlow) [#43927](nodejs/node#43927) - \[[`aac97c2d2a`](nodejs/node@aac97c2d2a)] - **deps**: update undici to 5.8.0 (Node.js GitHub Bot) [#43886](nodejs/node#43886) - \[[`cdff61917d`](nodejs/node@cdff61917d)] - **deps**: clean archs files for OpenSSL (RafaelGSS) [#43735](nodejs/node#43735) - \[[`fc936a84e4`](nodejs/node@fc936a84e4)] - **deps**: remove not used architectures (RafaelGSS) [#43735](nodejs/node#43735) - \[[`361a643d8b`](nodejs/node@361a643d8b)] - **deps**: V8: backport [`f3cad8c`](nodejs/node@f3cad8cec656) (Joyee Cheung) [#43531](nodejs/node#43531) - \[[`2e1732ebd0`](nodejs/node@2e1732ebd0)] - **deps**: V8: backport [`22698d2`](nodejs/node@22698d267667) (Chengzhong Wu) [#43751](nodejs/node#43751) - \[[`979f469d3a`](nodejs/node@979f469d3a)] - **deps**: upgrade npm to 8.15.0 (npm team) [#43917](nodejs/node#43917) - \[[`4096d81988`](nodejs/node@4096d81988)] - **deps**: upgrade npm to 8.14.0 (npm team) [#43826](nodejs/node#43826) - \[[`2ec8092e2c`](nodejs/node@2ec8092e2c)] - **deps,src**: use SIMD for normal base64 encoding (Brian White) [#39775](nodejs/node#39775) - \[[`67b4edde37`](nodejs/node@67b4edde37)] - **dns**: fix getServers return undefined (jiahao.si) [#43922](nodejs/node#43922) - \[[`7c75539a88`](nodejs/node@7c75539a88)] - **dns**: fix cares memory leak (theanarkh) [#43912](nodejs/node#43912) - \[[`1f80b88da5`](nodejs/node@1f80b88da5)] - **doc**: update email and mailmap for BethGriggs (Beth Griggs) [#43985](nodejs/node#43985) - \[[`8a2a6e16eb`](nodejs/node@8a2a6e16eb)] - **doc**: add 15.x - 18.x to Other Versions section (shhh7612) [#43940](nodejs/node#43940) - \[[`51cb0d42ca`](nodejs/node@51cb0d42ca)] - **doc**: inspector.close undefined in worker threads (Keyhan Vakil) [#43867](nodejs/node#43867) - \[[`c789c0f5f7`](nodejs/node@c789c0f5f7)] - **doc**: improve documentation for safe `Promise` statics alternatives (Antoine du Hamel) [#43759](nodejs/node#43759) - \[[`cb9b0e0011`](nodejs/node@cb9b0e0011)] - **doc**: recommend git-node-v8 (Keyhan Vakil) [#43934](nodejs/node#43934) - \[[`d7e9bd1830`](nodejs/node@d7e9bd1830)] - **doc**: clarify subprocess.stdout/in/err property (Kohei Ueno) [#43910](nodejs/node#43910) - \[[`808793ebb5`](nodejs/node@808793ebb5)] - **doc**: fix typo in `src/crypto/README.md` (Jianru Lin) [#43968](nodejs/node#43968) - \[[`bbc455c4f9`](nodejs/node@bbc455c4f9)] - **doc**: remind backporter about v8\_embedder_string (Keyhan Vakil) [#43924](nodejs/node#43924) - \[[`a86b66c8b4`](nodejs/node@a86b66c8b4)] - **doc**: fix typo in http.md (Airing) [#43933](nodejs/node#43933) - \[[`a96af37233`](nodejs/node@a96af37233)] - **doc**: add F3n67u to collaborators (Feng Yu) [#43953](nodejs/node#43953) - \[[`aa7d4e59f7`](nodejs/node@aa7d4e59f7)] - **doc**: improve test runner timeout docs (Tobias Nießen) [#43836](nodejs/node#43836) - \[[`80c2fa8212`](nodejs/node@80c2fa8212)] - **doc**: mention Win 32-bit openssl build issue (RafaelGSS) [#43853](nodejs/node#43853) - \[[`8b8c55df7e`](nodejs/node@8b8c55df7e)] - **doc**: add security release specifics to releases.md (Beth Griggs) [#43835](nodejs/node#43835) - \[[`42693aaf9f`](nodejs/node@42693aaf9f)] - **doc**: add history info for `global.performance` (Antoine du Hamel) [#43841](nodejs/node#43841) - \[[`140d6af572`](nodejs/node@140d6af572)] - **doc**: add platform-windows-arm to who to CC (Michael Dawson) [#43808](nodejs/node#43808) - \[[`976093efe3`](nodejs/node@976093efe3)] - **doc**: document ES2022's Error "cause" property (James Ide) [#43830](nodejs/node#43830) - \[[`ec7e45e4a2`](nodejs/node@ec7e45e4a2)] - **doc**: include make clean to openssl arch (RafaelGSS) [#43735](nodejs/node#43735) - \[[`d64dfd53c9`](nodejs/node@d64dfd53c9)] - **doc**: add link to diagnostic tools (Rafael Gonzaga) [#43736](nodejs/node#43736) - \[[`2910136920`](nodejs/node@2910136920)] - **doc**: update links to MDN page about dynamic imports (Jannis R) [#43847](nodejs/node#43847) - \[[`d88a9fae79`](nodejs/node@d88a9fae79)] - **doc**: deprecate coercion to integer in process.exit (Daeyeon Jeong) [#43738](nodejs/node#43738) - \[[`fc843e103d`](nodejs/node@fc843e103d)] - **doc**: add MoLow to triagers (Moshe Atlow) [#43799](nodejs/node#43799) - \[[`8c8c97da61`](nodejs/node@8c8c97da61)] - **(SEMVER-MINOR)** **doc**: deprecate diagnostics_channel object subscribe method (Stephen Belanger) [#42714](nodejs/node#42714) - \[[`9b53a694b5`](nodejs/node@9b53a694b5)] - **doc**: revert anachronistic 'node:' module require()s in API history notes (DeeDeeG) [#43768](nodejs/node#43768) - \[[`2815bd3002`](nodejs/node@2815bd3002)] - **doc**: clarify release process for new releasers (Rafael Gonzaga) [#43739](nodejs/node#43739) - \[[`50b3750e67`](nodejs/node@50b3750e67)] - **doc**: fix typo in ngtcp2 readme (Dan Castillo) [#43767](nodejs/node#43767) - \[[`6bcd40dd85`](nodejs/node@6bcd40dd85)] - **domain**: fix vm promise tracking while keeping isolation (Stephen Belanger) [#43556](nodejs/node#43556) - \[[`e89e0b470b`](nodejs/node@e89e0b470b)] - **esm**: remove superfluous argument (Rich Trott) [#43884](nodejs/node#43884) - \[[`0d2921f396`](nodejs/node@0d2921f396)] - **esm**: fix erroneous re-initialization of ESMLoader (Jacob Smith) [#43763](nodejs/node#43763) - \[[`9b5b8d78c3`](nodejs/node@9b5b8d78c3)] - **esm**: throw on any non-2xx response (LiviaMedeiros) [#43742](nodejs/node#43742) - \[[`dfc4832ef1`](nodejs/node@dfc4832ef1)] - **(SEMVER-MINOR)** **events**: expose CustomEvent on global with CLI flag (Daeyeon Jeong) [#43885](nodejs/node#43885) - \[[`e4473952ae`](nodejs/node@e4473952ae)] - **(SEMVER-MINOR)** **events**: add `CustomEvent` (Daeyeon Jeong) [#43514](nodejs/node#43514) - \[[`100f6deb09`](nodejs/node@100f6deb09)] - **fs**: use signed types for stat data (LiviaMedeiros) [#43714](nodejs/node#43714) - \[[`25ec71db63`](nodejs/node@25ec71db63)] - **http**: fix http server connection list when close (theanarkh) [#43949](nodejs/node#43949) - \[[`ca658c8afe`](nodejs/node@ca658c8afe)] - **(SEMVER-MINOR)** **http**: add drop request event for http server (theanarkh) [#43806](nodejs/node#43806) - \[[`9c699bd8a8`](nodejs/node@9c699bd8a8)] - **http**: wait for pending responses in closeIdleConnections (Paolo Insogna) [#43890](nodejs/node#43890) - \[[`781d5e54e3`](nodejs/node@781d5e54e3)] - **inspector**: set sampling interval before start (Shelley Vohr) [#43779](nodejs/node#43779) - \[[`0b5dbb2a56`](nodejs/node@0b5dbb2a56)] - **lib**: refactor PriorityQueue to use private field (Finn Yu) [#43889](nodejs/node#43889) - \[[`324473ca32`](nodejs/node@324473ca32)] - **(SEMVER-MINOR)** **lib**: improved diagnostics_channel subscribe/unsubscribe (Stephen Belanger) [#42714](nodejs/node#42714) - \[[`5aa3b213ac`](nodejs/node@5aa3b213ac)] - **meta**: update AUTHORS (Node.js GitHub Bot) [#43966](nodejs/node#43966) - \[[`e707552357`](nodejs/node@e707552357)] - **meta**: update `node-api` in label-pr-config (Daeyeon Jeong) [#43794](nodejs/node#43794) - \[[`8a8de94034`](nodejs/node@8a8de94034)] - **meta**: update AUTHORS (Node.js GitHub Bot) [#43872](nodejs/node#43872) - \[[`7d49fc766c`](nodejs/node@7d49fc766c)] - **meta**: use platform dropdown on flaky template (Rafael Gonzaga) [#43855](nodejs/node#43855) - \[[`e4aa50fc3f`](nodejs/node@e4aa50fc3f)] - **meta**: enable blank issues (Matteo Collina) [#43775](nodejs/node#43775) - \[[`ceb7c150ec`](nodejs/node@ceb7c150ec)] - **meta**: move one or more collaborators to emeritus (Node.js GitHub Bot) [#43770](nodejs/node#43770) - \[[`29bcd47738`](nodejs/node@29bcd47738)] - **net**: fix socket.\_getpeername (Daeyeon Jeong) [#43010](nodejs/node#43010) - \[[`380659daf1`](nodejs/node@380659daf1)] - **process**: use `defineProperty` instead of assignment (Mark S. Miller) [#43907](nodejs/node#43907) - \[[`aba9c8ebea`](nodejs/node@aba9c8ebea)] - **repl**: fix overzealous top-level await (Tobias Nießen) [#43827](nodejs/node#43827) - \[[`1deb6b73b7`](nodejs/node@1deb6b73b7)] - **repl**: use `SafePromiseAll` and `SafePromiseRace` (Antoine du Hamel) [#43758](nodejs/node#43758) - \[[`bf8f2e23ff`](nodejs/node@bf8f2e23ff)] - **src**: refactor DH groups to delete crypto_groups.h (Tobias Nießen) [#43896](nodejs/node#43896) - \[[`9435fbf8cd`](nodejs/node@9435fbf8cd)] - **src**: remove dead code in base64\_encode (Tobias Nießen) [#43979](nodejs/node#43979) - \[[`2c47e58ea0`](nodejs/node@2c47e58ea0)] - **src**: fix regression that a source marker is lost (cola119) [#43086](nodejs/node#43086) - \[[`d084150320`](nodejs/node@d084150320)] - **src**: per-isolate eternal template properties (Chengzhong Wu) [#43802](nodejs/node#43802) - \[[`9f9d00ccbb`](nodejs/node@9f9d00ccbb)] - **src**: merge NativeModuleEnv into NativeModuleLoader (Joyee Cheung) [#43824](nodejs/node#43824) - \[[`bb512904e9`](nodejs/node@bb512904e9)] - **src**: use named struct instead of typedef (Tobias Nießen) [#43881](nodejs/node#43881) - \[[`bb5511e8cc`](nodejs/node@bb5511e8cc)] - **src**: use named enum instead of typedef (Tobias Nießen) [#43880](nodejs/node#43880) - \[[`5db0c8f667`](nodejs/node@5db0c8f667)] - **src**: pass only Isolate\* and env_vars to EnabledDebugList::Parse() (Darshan Sen) [#43668](nodejs/node#43668) - \[[`249365524e`](nodejs/node@249365524e)] - **src**: fix node watchdog race condition (theanarkh) [#43780](nodejs/node#43780) - \[[`17cb27237d`](nodejs/node@17cb27237d)] - **src**: deduplicate `SetALPN` implementations (Tobias Nießen) [#43756](nodejs/node#43756) - \[[`b4c75a96be`](nodejs/node@b4c75a96be)] - **src**: fix `napi_check_object_type_tag()` (Daeyeon Jeong) [#43788](nodejs/node#43788) - \[[`8432d6596f`](nodejs/node@8432d6596f)] - **src**: slim down env-inl.h (Ben Noordhuis) [#43745](nodejs/node#43745) - \[[`2266a4b6d6`](nodejs/node@2266a4b6d6)] - **stream**: improve `respondWithNewView()` (Daeyeon Jeong) [#43866](nodejs/node#43866) - \[[`bf3991b406`](nodejs/node@bf3991b406)] - **stream**: fix 0 transform hwm backpressure (Robert Nagy) [#43685](nodejs/node#43685) - \[[`a057510037`](nodejs/node@a057510037)] - **stream**: initial approach to include strategy options on Readable.toWeb() (txxnano) [#43515](nodejs/node#43515) - \[[`198cf59d2c`](nodejs/node@198cf59d2c)] - **test**: update WPT encoding tests (Kohei Ueno) [#43958](nodejs/node#43958) - \[[`f0ed1aed8d`](nodejs/node@f0ed1aed8d)] - **test**: remove test-whatwg-events-add-event-listener-options-once.js (Feng Yu) [#43877](nodejs/node#43877) - \[[`88505556fe`](nodejs/node@88505556fe)] - **test**: work scheduled in process.nextTick can keep the event loop alive (Andreu Botella) [#43787](nodejs/node#43787) - \[[`81a21946eb`](nodejs/node@81a21946eb)] - **test**: simplify test-tls-set-secure-context (Tobias Nießen) [#43878](nodejs/node#43878) - \[[`61cd11a8a7`](nodejs/node@61cd11a8a7)] - **test**: use `common.mustNotMutateObjectDeep()` in fs tests (LiviaMedeiros) [#43819](nodejs/node#43819) - \[[`b1081dbe12`](nodejs/node@b1081dbe12)] - **test**: fix test http upload timeout (theanarkh) [#43935](nodejs/node#43935) - \[[`efd5e0e925`](nodejs/node@efd5e0e925)] - **test**: simplify ReplStream.wait() (Tobias Nießen) [#43857](nodejs/node#43857) - \[[`ef21ad2996`](nodejs/node@ef21ad2996)] - **test**: merge test-crypto-dh-hash with modp18 test (Tobias Nießen) [#43891](nodejs/node#43891) - \[[`e502c50a90`](nodejs/node@e502c50a90)] - **test**: refactor `test/es-module/test-esm-resolve-type` (Antoine du Hamel) [#43178](nodejs/node#43178) - \[[`c782c3dc69`](nodejs/node@c782c3dc69)] - **test**: ensure NODE_EXTRA_CA_CERTS not set before tests (KrayzeeKev) [#43858](nodejs/node#43858) - \[[`bb6787cb57`](nodejs/node@bb6787cb57)] - **test**: add check to test-fs-readfile-tostring-fail (Richard Lau) [#43850](nodejs/node#43850) - \[[`7571704186`](nodejs/node@7571704186)] - **test**: complete TODO in test/wpt/test-url.js (Kohei Ueno) [#43797](nodejs/node#43797) - \[[`6f1d2dfb9d`](nodejs/node@6f1d2dfb9d)] - **test**: add test on worker process.exit in async modules (Chengzhong Wu) [#43751](nodejs/node#43751) - \[[`776cc3abbd`](nodejs/node@776cc3abbd)] - **test**: use `common.mustNotMutateObjectDeep()` in immutability tests (LiviaMedeiros) [#43196](nodejs/node#43196) - \[[`42f2deb3a0`](nodejs/node@42f2deb3a0)] - **test**: add `common.mustNotMutateObjectDeep()` (LiviaMedeiros) [#43196](nodejs/node#43196) - \[[`f3fc51c508`](nodejs/node@f3fc51c508)] - **test**: fix coverity warning in test (Michael Dawson) [#43631](nodejs/node#43631) - \[[`a9ecba2fa8`](nodejs/node@a9ecba2fa8)] - **test**: mark test-http-client-response-timeout flaky (Tobias Nießen) [#43792](nodejs/node#43792) - \[[`cd0d9ddb7c`](nodejs/node@cd0d9ddb7c)] - **test_runner**: add support for boolean values for `concurrency` option (Lenvin Gonsalves) [#43887](nodejs/node#43887) - \[[`f98020138a`](nodejs/node@f98020138a)] - **test_runner**: validate `timeout` option (Antoine du Hamel) [#43843](nodejs/node#43843) - \[[`58d15b3687`](nodejs/node@58d15b3687)] - **test_runner**: pass signal on timeout (Moshe Atlow) [#43911](nodejs/node#43911) - \[[`8b0248506f`](nodejs/node@8b0248506f)] - **test_runner**: do not report an error when tests are passing (Antoine du Hamel) [#43919](nodejs/node#43919) - \[[`aa8053e1fa`](nodejs/node@aa8053e1fa)] - **test_runner**: recieve and pass AbortSignal (Moshe Atlow) [#43554](nodejs/node#43554) - \[[`f13e4c1be9`](nodejs/node@f13e4c1be9)] - **test_runner**: fix `it` concurrency (Moshe Atlow) [#43757](nodejs/node#43757) - \[[`e404a3ef6d`](nodejs/node@e404a3ef6d)] - **test_runner**: support timeout for tests (Moshe Atlow) [#43505](nodejs/node#43505) - \[[`f28198cc05`](nodejs/node@f28198cc05)] - **test_runner**: catch errors thrown within `describe` (Moshe Atlow) [#43729](nodejs/node#43729) - \[[`bfe0ac6cd0`](nodejs/node@bfe0ac6cd0)] - **tools**: add more options to track flaky tests (Antoine du Hamel) [#43954](nodejs/node#43954) - \[[`17a4e5e775`](nodejs/node@17a4e5e775)] - **tools**: add verbose flag to inactive TSC finder (Rich Trott) [#43913](nodejs/node#43913) - \[[`373304b0c7`](nodejs/node@373304b0c7)] - **tools**: add support for using API key to vuln checking script (Facundo Tuesca) [#43909](nodejs/node#43909) - \[[`ed45088c14`](nodejs/node@ed45088c14)] - **tools**: support versioned node shared libs on z/OS (alexcfyung) [#42256](nodejs/node#42256) - \[[`c9ecd6d21f`](nodejs/node@c9ecd6d21f)] - **tools**: update doc to [email protected] (Node.js GitHub Bot) [#43870](nodejs/node#43870) - \[[`c92135aa0f`](nodejs/node@c92135aa0f)] - **tools**: update lint-md-dependencies to [email protected] (Node.js GitHub Bot) [#43871](nodejs/node#43871) - \[[`e12bf40fd1`](nodejs/node@e12bf40fd1)] - **tools**: update eslint to 8.20.0 (Node.js GitHub Bot) [#43873](nodejs/node#43873) - \[[`09fe9b30a9`](nodejs/node@09fe9b30a9)] - **tools**: add script for vulnerability checking (Facundo Tuesca) [#43362](nodejs/node#43362) - \[[`19e8876877`](nodejs/node@19e8876877)] - **trace_events**: trace net connect event (theanarkh) [#43903](nodejs/node#43903) - \[[`1af7f24143`](nodejs/node@1af7f24143)] - **util**: remove unicode support todo for perf implications (Rhys) [#43762](nodejs/node#43762) - \[[`acfc33ca8c`](nodejs/node@acfc33ca8c)] - **(SEMVER-MINOR)** **util**: add tokens to parseArgs (John Gee) [#43459](nodejs/node#43459) - \[[`f32aec8a6d`](nodejs/node@f32aec8a6d)] - **util**: refactor to use validateObject (Kohei Ueno) [#43769](nodejs/node#43769) - \[[`d7cfd0c5ba`](nodejs/node@d7cfd0c5ba)] - **v8**: serialize BigInt64Array and BigUint64Array (Ben Noordhuis) [#43571](nodejs/node#43571)

PR-URL: nodejs#43685 Refs: nodejs#42457 Refs: https://github.com/nodejs/node/pull/43648/files Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]>

PR-URL: nodejs/node#43685 Refs: nodejs/node#42457 Refs: https://github.com/nodejs/node/pull/43648/files Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]>

ronag and others added 2 commits July 5, 2022 10:57

stream: fix 0 transform hwm backpressure

446aba6

stream: add 0 hwm test

67eaa2f

ronag added the stream Issues and PRs related to the stream subsystem. label Jul 5, 2022

ronag requested a review from mcollina July 5, 2022 08:58

nodejs-github-bot added the needs-ci PRs that need a full CI run. label Jul 5, 2022

ronag changed the title ~~Fix 42457~~ stream: fix 0 transform hwm backpressure Jul 5, 2022

This comment was marked as resolved.

Sign in to view

ronag marked this pull request as draft July 5, 2022 09:46

ronag added 2 commits July 5, 2022 12:49

fixup! stream: fix 0 transform hwm backpressure

4688771

fixup! stream: fix 0 transform hwm backpressure

7b758da

ronag marked this pull request as ready for review July 5, 2022 10:50

ronag added 6 commits July 5, 2022 12:52

fixup! stream: fix 0 transform hwm backpressure

150c51b

fixup! stream: fix 0 transform hwm backpressure

2d5a05f

fixup! stream: fix 0 transform hwm backpressure

63e4e53

fixup! stream: fix 0 transform hwm backpressure

ac2c5b3

fixup! stream: fix 0 transform hwm backpressure

3654d6e

fixup! stream: fix 0 transform hwm backpressure

386f2ec

mcollina approved these changes Jul 5, 2022

View reviewed changes

benjamingr approved these changes Jul 5, 2022

View reviewed changes

benjamingr reviewed Jul 5, 2022

View reviewed changes

lib/internal/streams/transform.js Show resolved Hide resolved

ronag added the request-ci Add this label to start a Jenkins CI on a PR. label Jul 11, 2022

ronag force-pushed the fix-42457 branch from eb930d7 to b426a13 Compare July 11, 2022 09:51

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 11, 2022

This was referenced Jul 13, 2022

CI Reliability 2022-07-13 nodejs/reliability#321

Open

CI Reliability 2022-07-14 nodejs/reliability#322

Open

github-actions bot mentioned this pull request Jul 15, 2022

CI Reliability 2022-07-15 nodejs/reliability#323

Open

24 tasks

mcollina approved these changes Jul 18, 2022

View reviewed changes

nodejs-github-bot added commit-queue-failed An error occurred while landing this pull request using GitHub Actions. and removed commit-queue Add this label to land a pull request using GitHub Actions. labels Jul 18, 2022

aduh95 merged commit 31d9edc into nodejs:main Jul 24, 2022

danielleadams mentioned this pull request Jul 26, 2022

v18.7.0 proposal #43993

Merged

targos mentioned this pull request Aug 3, 2022

v16.17.0 release proposal #44098

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

stream: fix 0 transform hwm backpressure #43685

stream: fix 0 transform hwm backpressure #43685

ronag commented Jul 5, 2022 •

edited

Loading

nodejs-github-bot commented Jul 5, 2022

ronag commented Jul 5, 2022

This comment was marked as resolved.

mcollina left a comment

kanongil commented Jul 5, 2022 •

edited

Loading

kanongil commented Jul 5, 2022

ronag commented Jul 5, 2022 •

edited

Loading

kanongil commented Jul 5, 2022

kanongil commented Jul 5, 2022 •

edited

Loading

benjamingr left a comment

kanongil commented Jul 5, 2022

benjamingr commented Jul 5, 2022

ronag commented Jul 5, 2022

nodejs-github-bot commented Jul 11, 2022

nodejs-github-bot commented Jul 12, 2022

nodejs-github-bot commented Jul 14, 2022

nodejs-github-bot commented Jul 17, 2022

mcollina left a comment

nodejs-github-bot commented Jul 18, 2022

mcollina commented Jul 18, 2022

aduh95 commented Jul 24, 2022

aduh95 commented Jul 24, 2022

stream: fix 0 transform hwm backpressure #43685

stream: fix 0 transform hwm backpressure #43685

Conversation

ronag commented Jul 5, 2022 • edited Loading

nodejs-github-bot commented Jul 5, 2022

ronag commented Jul 5, 2022

This comment was marked as resolved.

mcollina left a comment

Choose a reason for hiding this comment

kanongil commented Jul 5, 2022 • edited Loading

kanongil commented Jul 5, 2022

ronag commented Jul 5, 2022 • edited Loading

kanongil commented Jul 5, 2022

kanongil commented Jul 5, 2022 • edited Loading

benjamingr left a comment

Choose a reason for hiding this comment

kanongil commented Jul 5, 2022

benjamingr commented Jul 5, 2022

ronag commented Jul 5, 2022

nodejs-github-bot commented Jul 11, 2022

nodejs-github-bot commented Jul 12, 2022

nodejs-github-bot commented Jul 14, 2022

nodejs-github-bot commented Jul 17, 2022

mcollina left a comment

Choose a reason for hiding this comment

nodejs-github-bot commented Jul 18, 2022

mcollina commented Jul 18, 2022

aduh95 commented Jul 24, 2022

aduh95 commented Jul 24, 2022

ronag commented Jul 5, 2022 •

edited

Loading

kanongil commented Jul 5, 2022 •

edited

Loading

ronag commented Jul 5, 2022 •

edited

Loading

kanongil commented Jul 5, 2022 •

edited

Loading